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ABSTRACT 



A data processing system having a pair of mirrored 
storage units maintains a state record of the mirrored 
pair in system memory. In order to be able to determine 
state when the system is re-initialized, this state informa- 
tion is also stored on each storage unit of the mirrored 
pair, and in an alternate location. When the state 
changes, the operating system writes the new state to 
those storage units which are still functioning, and to 
the alternate location. In order to prevent ambiguous 
situations, only certain defined state transitions are per- 
mitted. When the system is re-initialized, it attempts to 
read the state information stored on the storage units. If 
either unit can not be read, the system substitutes the 
state retrieved from the alternate state record for the 
state that would have been read from the non-respond- 
ing unit This pair of states from the two units index an 
unique entry in a state derivation table containing the 
resultant state. 

16 Claims, 10 Drawing Sheets 



C HAIN ^ 

I I PL TASK J 





171 


* 


/ 


FIRST PART OF 


RE-INITIALIZATION 




r 172 




ALLOW I PL 
TO CONTINUE 



.171 



HANDLE ERROR 
CONDITION 



( ST0P ) 



08/27/2003, EAST Version: 1.04.0000 



U.S. Patent 



July 4, 1995 



Sheet 1 of 10 



5,430,866 



100 



\ 



103 



105 



MAIN 
MEMORY 



OPERATING 
SYSTEM 




DEVICE 

CONFIG. 

TABLE 




STATE 

DERIVATION 
TABLE 



106 



101 



104 



SYSTEM 
PROCESSOR 



115- 



N-V RAM 



• 






SYSTEM SERIAL # 




-107 


NVRAM SERIAL # 




-108 


LOAD SOURCE STATE 




"109 


UNIT 'A' ADDR. 




-no 


UNIT 'B' ADDR. 




-111 


• 
• 








- ^ J_OGICAL_ ID_ _ 
v SYSTEM SERIAL J. 
v JIVRAMTERIAL #. 
DEVICE CORF fiT. "TABLE 



143 



SYSTEM SERIAL JL. <r> 



v J(yRAM"§ERIAL #. 
DEVICE CORFfG. "TABLE. 



•-^2-143 



FIG. I 



08/27/2003, EAST Version: 1.04.0000 



U.S. Patent 



July 4, 1995 



Sheet 2 of 10 




08/27/2003, EAST Version: 1.04.0000 



U.S. Patent 



July 4, 1995 



Sheet 3 of 10 



5,430,866 




08/27/2003, EAST Version: 1.04.0000 



U.S. Patent July 4, 1995 Sheet 4 of 10 5,430 



START 
DEVICE CONFIG 
TASK 



z 



402 



READ LOGICAL ID, 
SYSTEM SERIAL ft 
FROM PRIMARY 
LOAD SOURCE 



JL 



403 



READ DCT INTO 
MAIN MEMORY 



404 



410 



CALL RETRIEVE 
MIRRORED UNIT 
STATE (FIG. 4B) 



READ DISK UNIT ID 
FROM DISK UNIT 



I 



WAIT FOR NEXT DISK 
UNIT TO REPORT 



I 




RE- I PL SYSTEM 
FROM ALTERNATE 
LOAD SOURCE UNIT 



FIG. 4A 



08/27/2003, EAST Version: 1.04.0000 



U.S. Patent July 4, 1995 Sheet 5 of 10 5,430, 



c 



RETRIEVE MIRRORED 
UNIT STATE 



) 



422 




CALL VALIDATE 
DCT (FIG. 4F) 




RETURN STATE 
VALUE FROM DCT 
FOR SPECIFIED 
MIRRORED PAIR 



L 



YES 




CALL GET LOAD 


— p 


SOURCE STATE 






(FIG. HC) 



( RETURN ) 



426 



jL 



RETURN STATE 
'UNKNOWN' 




FIG. 4B 



08/27/2003, EAST Version: 1.04.0000 



U.S. Patent July 4, 1995 sheet 6 of io 5,430,866 



c 



GET LOAD 
SOURCE STATE 



431 

Z_ 



CALL VALIDATE 
NVRAM (FIG. AD) 



432 



CALL GET 
STORED STATE 
(UNIT A) 
(FIG, 4E) 



433 



CALL GET 
STORED STATE 
(UNIT B) 
(FIG. 4E) 



434 



LOOK UP LOAD SOURCE 
STATE IN STATE 
DERIVATION TABLE 
(SEE FIG. 3) 



C 



I 



RETURN 



5 



FIG. 4C 



08/27/2003, EAST Version: 1.04.0000 



U.S. Patent July 4, 1995 sheet 7 of io 5,430,866 



C VALIDATE ) 
^ NVRAM J 



I 



441 



COMPARE NVRAM WITH 
DATA FROM PRIMARY 
LOAD SOURCE UNIT 




FIG. 4D 

08/27/2003, EAST Version: 1.04.0000 



U.S. Patent 



July 4, 1995 



Sheet 8 of 10 



5,430,866 



YES 











RETURN STATE READ 
FROM DISK UNIT 
WHEN IT REPORTED 










RETURN STATE 
'UNKNOWN' 



( RETURN y 



H55 
A 

RETURN STATE 
STORED IN 
NVRAN 



FIG. 4E 



08/27/2003, EAST Version: 1.04.0000 



U.S. Patent July 4, 1995 Sheet 9 of 10 5,430,866 




YES462 



CALL GET LOAD 
SOURCE STATE 
(FIG. 1C) 



.463 

PRIMARY 
LOAD SOURCE UNIT 
CONTAINS CURRENT 
DATA? 



NO 



SET DCT 
NOT USABLE 



SET DCT 
USABLE 







^ RETURN J 



FIG. 4F 



08/27/2003, EAST Version: 1.04.0000 



U.S. Patent July 4, 1995 sheet 10 of 10 5,430,866 



c 



MAIN 

I PL TASK 



471 

/ 



FIRST PART OF 
RE- INITIALIZATION 




ALLOW 


IPL 


TO CONTINUE 







" C STOP J 



z 



474 



HANDLE ERROR 
CONDITION 



( ST ° P ) 



FIG. 4G 



08/27/2003, EAST Version: 1.04.0000 



METHOD AND APPARATUS FOR DERIVING 
MIRRORED UNIT STATE WHEN 
RE-INITIALIZING A SYSTEM 

5 

This application is a continuation of application Ser. 
No. 07/522,345 filed May 11, 1990, now abandoned. 

BACKGROUND OF THE INVENTION 

The present invention relates to maintaining mirrored 10 
copies of computer data in redundant data storage units, 
and in particular to determining which units contain 
current data when re-initializing the system. 

The extensive data storage needs of modern com- 
puter systems require large capacity mass data storage 15 
devices. A common storage device is the magnetic disk 
drive, a complex piece of machinery containing many 
parts which are susceptible to failure. A typical com- 
puter system will contain several such units. As users . 
increase their need for data storage, systems are config- 20 
ured with larger numbers of storage units. The failure of 
a single storage unit can be a very disruptive event for 
the system. Many systems are unable to operate until 
the defective unit is repaired or replaced, and the lost 
data restored. An increased number of storage units 25 
increases the probability that any one unit will fail, 
leading to system failure. At the same time, computer 
users are relying more and more on the consistent avail- 
ability of their systems. It therefore becomes essential to 
find improved methods of sustaining system operations 30 
in the presence of a storage unit failure, and restoring 
the system to normal operating mode when the failure 
condition has been corrected. 

One method of addressing these problems is known as 
"nmToring , \ This method involves maintaining a dupli- 35 
cate set of storage devices, which contains the same 
data as the original. The duplicate set is available to 
assume the task of providing data to the system should 
any unit in the original set fail. A system may have a 
duplicate set of all stored data ("fully mirrored")t or of 40 
some subset of the data ("partially mirrored"). Mirror- 
ing is becoming increasingly attractive as computer 
users demand improved system reliability and availabil- 
ity. 

A user with a system containing mirrored storage 45 
will expect the utmost in reliability from his storage 
Since the essence of mirroring is that if one storage unit 
fails, another is available to take its place, the system 
must necessarily be able to operate with only one of a 
pair of mirrored units functioning. When both units of a 50 
mirrored pair are functioning and contain current data, 
the units are said to be synchronized. If one of a mir- 
rored pair of storage units fails, and the other continues 
to operate, the data in the failing unit will soon become 
obsolete. The "failure" of a unit simply means that data 55 
can no longer be read from or written to the unit This 
could mean that the storage unit itself is not operating, 
or that some other component of the system, such as an 
I/O processor, is not functioning. Restoring the failing 
storage unit to operation may leave the data on the 60 
storage medium intact, as when a circuit card contain- 
ing control logic is replaced. 

Because a system may operate when the disk units of 
a mirrored pair are no longer synchronized, it must 
know the state of the mirrored pau\ i.e.,, which unit or 65 
units contain current data. If the system is powered 
down for any reason, it must be able to reconstruct the 
state of its storage units when power is restored and the 
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system re-initializes itself. If a failing storage unit was 
repaired or replaced while the system was down, upon 
re-initialization the operating system must be able to 
ascertain that data contained on the repaired or re- 
placed unit is unreliable, and initiate a process to re-syn- 
chronize the units, which brings the data on the re- 
paired or replaced unit current with that on the non-fail- 
ing (current) unit. 

One method of ascertaining the state of a mirrored 
pair of storage units is to store state information on both 
units. On re-initialization, the system reads this state 
information. If both units are functioning and the stored 
state information on both units is that they are synchro- 
nized with each other, the system determines that this is 
the case. In the event of a single storage unit failure 
while the operating system is up and running, where all 
other devices operate properly, the operating system 
will recognize that the non-failing unit alone has current 
data, and record this new state information on the non- 
failing unit When re-initialized after repair, the state 
information on the non-failing unit will be that it alone 
has current data, while the failing unit's state record 
may indicate that both units are synchronized or some 
unknown state. The operating system is able to deter- 
mine in this situation that only the non-failing unit con- 
tains current data. 

However, during re-initialization of the system, it is 
not uncommon for one of a mirrored pair of units to 
report that both units are synchronized, while the other 
unit does not respond. In this case, the system can not 
determine the state of the mirrored pair with certainty. 
It is possible that both units were synchronized when 
the system was powered down, as claimed by the re- 
sponding unit But the same situation can arise, for ex- 
ample, when the 'A 1 unit fails, is repaired without loss of 
its obsolete data, the system is re-initialized, and the *B' 
unit does not respond. Note that a failure to respond 
during re-initialization does not necessarily mean that a 
storage unit is broken. The power switch may be off, or 
any number of other circumstances may prevent the 
unit from responding, particularly where a repair action 
has taken place while the system was powered down. 

In the above mentioned situations, the operating sys- 
tem will either be unable to make a state determination, 
or will guess, possibly making an incorrect state deter- 
mination. If the operating system is unable to make a 
state determination, it will generally query the user for 
the correct state. Because there may be a large number 
of storage units, and the association of logical address to 
physical location will not necessarily be obvious, query- 
ing the user is a very unreliable method of determining 
state. Guessing the state or just not knowing the state 
are both clearly undesirable for a mirrored or fault 
tolerant computer system, since the user does not re- 
ceive the reliability and availability he expects. 

It is therefore an object of the present invention to 
provide an enhanced method and apparatus for deter- 
mining the state of a mirrored pair of data storage units. 

It is a further object of this invention to provide an 
enhanced method and apparatus for determining the 
state of a mirrored pair of data storage units where 
multiple device failures occur. 

It is also an object of this invention to provide greater 
redundancy and reliability in information tracking the 
state of mirrored storage units of a data processing 
system. 

Another object of this invention is to provide a 
method and apparatus for determining the state of a 
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mirrored pair of data storage units which is less prone to I/O processors 120,130. Main memory 103 is coupled to 
human error. system bus 102 and directly to system processor 101. 

„ rn „, A „^ r ^ ____ m» w«t/»» t During normal operations, operating system 105 con- 

SUMMARY OF THE INVENTION taming Device Configuration Tabic 106 and State Deri- 

A data processing system having a pair of mirrored 5 vation Table 116 resides in main memory 103. Non- 
storage units maintains a record of the state ("state volatile RAM 104, containing system critical data, is 
record") of the mirrored pair in system memory. In coupled to system processor 101. The non-volatile 
order to be able to determine state when the system is RAM 104 contains alternate state record 115 of the 
re-initialized, this state information is also stored in mirrored unit state of the load source units. Alternate 
separate state records, one on each storage unit of the 10 state record 115 comprises a system serial number 107, 
mirrored pair. The state of the mirrored pair is also the mirrored unit state of the load source units 109, the 
stored in an alternate state record in an alternate loca- I/O address of load source unit 'A' 110, and the I/O 
tion. In a preferred embodiment, the alternate state address of load source unit 'B* 111. Non-volatile RAM 
record is stored in non-volatile RAM. When the state 104 may optionally contain the serial number of the 
changes, the operating system writes the new state to 15 non-volatile RAM circuit card itself 108. The I/O pro- 
those storage units which are still functioning, and to cessors 120,130 are attached to one or more storage 
the alternate state record. In order to prevent ambigu- units 121,122,131,132. In the preferred embodiment, 
ous situations, only certain defined state transitions are storage units 121,122,131,132 are magnetic disk drive 
permitted. units, but could be any storage unit such as optical disk, 

When the system is re-initialized, it attempts to read 20 tape, etc. Storage unit *A' 121 and storage unit 4 B' 131 
the state information stored on the storage units. If constitute a single rnirrored pair, and contain storage for 
either unit can not be read, the system retrieves the state the same data. In the preferred embodiment, unit * A* 
from the alternate record, and substitutes the state re- 121 and unit 4 B' 131 are attached to separate I/O proces- 
trieved from the alternate state record for the state that sors 120 and 130, thereby increasing system redun- 
would have been read from the non-responding unit 25 dancy. However, the units could be attached to the 
The pair of states read from the two units (or substituted same processor. Although two I/O processors 120,130 
from the alternate state record) index a unique entry in are shown, it should be understood that the system may 
a state derivation table. The entry contained at the loca- contain more than two I/O processors, or may contain 
tion indexed by the two retrieved states is the true cur- only one I/O processor. It should also be understood 
rent state of the mirrored pair of storage units, which is 30 that the number of storage units attached to an I/O 
then loaded into memory for system operation. processor is variable. In the preferred embodiment, 

In a preferred embodiment, the system may contain computer system 100 is an IBM AS/400 computer sys- 

multiple pairs of mirrored storage units. One pair is tern, altho ugh any comp uter system could be used. 

designated the "load source", and contains data neces- ^Eaclfstorage unit 12(122,131,132 contains a logicar 
sary for re-initializing the system. Each unit of the load 35 r identifier 140Fa=system^sOTal^umber-tt 
source pair will contain a state record called a "Device \record ca lteka .Device- Configur ation Table 143. Logf- 
Configuration Table". The Device Configuration Table cal"iderttifieT140 identifies the disk unit to the system^ 
on the load source units contains the states of all config- System-serial-number--141-is-the-serial-number-of-the 
ured storage units on the system. The alternate state system to which the unit is attached. In an alternate 
record contains the state of the load source pair, but not 40 embodiment, the disk units also contain the serial num- 
of the other storage units. When the system is re-initial- ber 142 of the non-volatile RAM circuit card 104. 
ized, it determines the state of the load source pair as The Device Configuration Table is shown in more 
described in the preceding paragraph. Once the state of detail in FIG. 2. It contains information identifying the 
the load source pair is known, the states of the remain- storage configuration and state of the storage units, 
ing devices can be obtained directly from the Device 45 Each storage unit contains a complete Devic e Config u- 
Configuration Table on any current load source unit. ration T ablcT Having entries for all the storage unite 

/it^da^W^T^M^WMle^ 201,202 in the 

BRIEF DESCRIPTION OF THE DRAWINGS / T g| W^^^S^^^^m^mmW^ a 

FIG. 1 is a block diagram of a system incorporating 4inglejr^ 
the mirrored storage components of the present inven- SO^cal unit number 210 onh^minxjred'pair OT smgle'un- 
tion; ^Tuirrored unitrln thepreferred embodiment, logical unit 

FIG. 2 is a diagram of a Device Configuration Table number 1 is reserved for the load source pair or unit ? 
contained on one of the storage units or in main mem- The remaining logical unit numbers are arbitrary. ^The 
ory; lpgicalunit humbeTTwhen^concaten 

FIG. 3 is the State Derivation Table used to deter- 55 /fr-desjignatkm, c*ik^^ 
mine the state of a mirrored pair of storage units in disk"unitTm~addrtion-to~the~logical unit number 210p 
accordance with this invention; 4ach record is.divMedinto three subrecords as shown^m 

FIGS. 4A-4G are a flow diagram of the steps re- FIG. 2: a common data subrecord 211, a subrecord 212 
quired to determine the state of the storage units at- for data peculiar to unit 44 A" of the pair, and a subrecord 
tached to the system when the system is re-initialized. 60 213 for data peculiar to unit "B". The common data 
_ __ ^n^^^, ^„ WTX , subrecord 211 contains a mirrored flag 215 and a mir- 

DE I^™^P^^^^^J HE rored unit state 214, The mirrored flag 215 is a single bit 

PREFERRED EMBODIMENT field used tQ whether the record is for a pair of 

A block diagram of the major components of com- mirrored disk units or a single unmirrored unit; a T 
puter system 100 of the preferred embodiment of the 65 indicates that the record is a pair of mirrored units. The 
present invention is shown in FIG. 1. A system proces- rnirrored unit state 214 is a one-byte field indicating the 
sor 101, suitable programmed as shown in FIG. 4, com- current state of the mirrored pair. If the rnirrored flag is 
municates over the system bus 102 with one or more set to 4 0' (not mirrored), the mirrored unit state is ig- 
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nored. In the example shown in FIG. 2, record 201 for When computer system 100 is re-initialized, it first 

logical unit number 1 is of a mirrored pair of disk units, derives the mirrored unit state of the load source pair 

in which the current state is both units are functioning from the mirrored state data on the two load source 

normally; record 202 for logical unit number 3 is of a units and in non-volatile RAM. A state is associated 

single unmirrored disk unit Since each unit number has 5 with each of the load source units. The state "associated 

its own entry in the Device Configuration Table indi- with" a load source unit is the state stored on that unit 

eating whether it is one of a miiTored pair, it is possible if it can be read, or the state stored in alternate state 

to mirror all, some or none of the disk units on a particu- record 115 in non-volatile RAM 104 if the state stored 

lar system. The subrecords 212,213 for unit * A* and unit on the storage unit can not be read. There exists a 

'B* each contain an I/O address field 216,218 and a unit 10 unique resultant state of the mirrored load source pair 

serial number 217,219 of their respective disk units. If corresponding to each pair of a state associated with the 

the record is of a single unmirrored disk unit, as shown 'A' unit and a state associated with the V unit. This 

for record 202, the information for the unit is stored in resultant state is the true state of the load source pair, 

the unit 'A' subrecord 212, and the unit 'W subrecord S * te Derivation Table 301 shown 

213 is not used. It should be understood that subrecords 15 m ^p'^The State Derivation Table 301, which is 

211, 212 and 213 may contain additional fields used for 116 in the operating system 105 in mam memory 

other purposes 103, requires an input associated with load source unit 

Therein- five possible states which may be stored in * m and an input associated with load source unit B 

the mirrored unkstate field 214 of the Device Configu- ,„ 303. The system derives the state of tte^ored load 

ration Table. These states and their associated meanings 20 ^ b * * e " ^f^JS 

column corresponding to the state associated with unit 

^BOK-Both -A' and V units of the mirrored pair A f d row corresponding to the state associated 

™ . , " , " F with unit B. The input states marked Unknown* 

^^^a^ . CUrr ?- d ♦ „ ♦ * 304,305 are not read from the disk unit or non-volatile 

AOK-Only the A unit contains current data; the ^ ^ but m used only when neither thc ^ ^ nor 

* "J f unable. non-volatile RAM can be read. The entries in the table 

BOK-Only the V unit contains current data; the marked ^ ^ «♦„ „ not possible. The entries 

^i^* 1 "* 1 ?- , marked "*AOK" and "*BOK" are not possible when 

AOKRB— The A' umt contains current data, and the usin ^ ^ read from ^ disk ^ but m 

TJ' unit is in the process of being re-synchromzed 3(J when ^ ^ ^ from non _ V olatile RAM for a non- 

with the 'A' umt responding disk unit. The entries marked "*»AOK" and 

BOKRA — The *B' unit contains current data, and the «**BOK" occur only for the state transitions from 

'A* unit is in the process of being re-synchronized BOKRA to ABOK with B failing or from AOKRB to 

with the *B' unit ABOK with A failing, respectively; computer system 

The system will not store an "unknown" state in the 35 X9Q recognizes and handles this 2-step transition, 

mirrored unit state field. As an example of the use of State Derivation Table 

During normal system operation, the operating sys- if unit *A' contains the state 'ABOK', and unit *B' 

tern maintains a copy of the Device Configuration does not respond, and non-volatile RAM contains the 

Table in main memory 103 (FIG. 1), to which it refers state «BOK\ the system will substitute the state 'BOK' 

when state information regarding the storage units is 40 read ft om non-volatile RAM for the state that would 

needed. When there is any change in the state of a stor- nave \ yccn rca d fx 0m the 4 B' unit Putting these values 

age unit, the new state is written to the Device Configu- mto tne table, the resultant state is contained at column 

ration Table in memory and the Device Configuration 'ABOK* and row 4 BOK\ This resultant state is *BOK\ 

Table on each operational storage unit. If the state This \ s the situation results when the *A' unit fails, its 

change involves a load source disk unit, the new state is 45 data becomes obsolete, the system is powered down, the 

also written to non-volatile RAM 104. In order to pre- «a* unit is restored to operation, and the *B* unit fails to 

vent the system from going into an indeterminate state, respond when the system is re-initialized, 

the following rules must be observed when changing FIGS. 4A-4G depict in detail the steps of the re-ini- 

states: tialization procedure ("IPL") according to the present 

1. The system must never allow both units of a mir- 50 invention. FIG. 4A shows the device configuration 
rored pair to be back level. If both units fail, the task. This process begins by reading the storage unit 
system shuts itself down. In this case, the last unit logical identifier from one of the load source units 402. 
to fail will contain current data. The load source unit which is accessed in this initial step 

2. The system must never attempt to write a new is herein designated the primary load source unit The 
mirrored state to a disk unit that fails. 55 primary load source unit in the preferred embodiment is 

3. Only the following state transitions are permissible: the unit which responds first to a poll, and could be 
ABOK to AOK — (B unit fails) either the logical *A' or logical 'B' unit The process 
ABOK to BOK— (A unit fails) then loads the Device Configuration Table ("DCT") 
AOKRB to AOK13 (Attempt to re-synchronize B from the primary load source unit into main memory at 

unit fails) 60 403. The process then enters a loop at 404 to determine 

BOKRA to BOK— (Attempt to re-synchronize A the state of all units. Storage units report in a random 

unit fails) pattern as they are brought on line. The process waits 

AOK to AOKRB— (Start re-synchronizing B unit) for each unit to report in 408, obtains the logical identi- 

BOK to BOKRA — (Start re-synchronizing A unit) flex of the reporting unit at 409, and calls a Retrieve 

AOKRB to ABOK — (Successful re-synchroniza- 65 Mirrored Unit State routine at 410 if the unit is one of a 

tion of B unit) mirrored pair 405. 

BOKRA to ABOK— (Successful re-synchroniza- The Retrieve Mirrored Unit State routine 410 returns 

tion of A unit) the state of the reporting unit based on the information 
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then available to the system. On return from the routine i.e., at least one storage unit containing current data for 

410, if both load source units have reported 411, and it all the stored data on the system. If so, IPL is allowed to 

the called subroutines have returned a state for the load continue 473; otherwise, an error handling routine is 

source pair indicating that the primary load source does called. 

not contain current data 412, the system will re-start its 5 By way of example, if the system is functional in all 

re-initialization procedure at 413, using the alternate respects and both load source units contain current 

load source unit. Otherwise, it will return to the loop data, but the 'B' units is powered off, the system would 

404. perform the following steps. It would enter the Device 

The retrieve mirrored unit state routine 410 is shown Configuration Task and read the identifier, system serial 

in detail in FIG. 4B. The routine branches 421 to a Get 10 402 and DCT 403 from unit 'A*. Since unit 'A* is mir- 

Load Source State routine 422 if the unit is a load rored 405, it would call Retrieve Mirrored Unit State 

source unit; otherwise it calls a Validate Device Config- 410. It would branch at 421 to call Get Load Source 

uration Table routine 423. If the Validate DCT routine State 422. Get Load Source State would call Validate 

returns DCT valid 424, the retrieve mirrored unit state NVRAM 431, which would set NVRAM valid and 

routine 410 returns the state value of the mirrored pair 15 return. It would then call Get Stored State for unit A 

stored in the DCT 425; otherwise it returns state not 432, which would return the state of 'ABOKL'. It would 

known 426. then call Get Stored State for unit B 433. Because unit 

The Get Load Source State routine called at 422 is B has not reported in 451, and NVRAM is usable 453, it 

shown in FIG. 4C. It successively calls a Validate will return the state in NVRAM, which will be 

NVRAM routine 431, a Get Stored State routine 432 20 *ABOK\ Get Load Source State would then look up 

for disk unit A, a Get Stored State routine for disk unit the resultant state in the State Derivation Table 434, 

B 433, and accesses the State Derivation Table 301 to which would be 'ABOK*. It then returns to Retrieve 

derive the load source state from the states returned Mirrored Unit State (FIG. 4B), which returns to the 

from the two calls to the Get Stored State routine. device configuration task (FIG. 4A). Since both load 

These returned state values are the states associated 25 source units have not reported 411, the task returns to 

with the load source A and B units. the loop 404 and waits for the next unit to report 408. It 

The Validate NVRAM routine is shown in FIG. 4D. will then continue receiving status from the re m a inin g 

This routine performs simple validity checks and sets a units. After sufficient time has elapsed, the Main IPL 

flag indicating NVRAM usable or not usable. One rea- Task, which has been performing other work 471, will 

son for performing this validation is that the non- 30 check whether there is at least one current unit for all 

volatile RAM unit (in this case, a circuit card) may have data on the system 472. Since the 'A' unit has been 

been replaced; it is for this reason that it is imprudent to determined to be current, it will allow IPL to continue 

rely on NVRAM alone to determine the state of the 473. 

load source units. The routine compares the data in In the preferred embodiment, the system is designed 

NVRAM with the data read from the primary load 35 so that any arbitrary set of storage units may be mir- 

source disk unit 441. If the NVRAM and the primary rored. It is, for example, possible to mirror some of the 

load source unit do not contain the same system serial units, but not mirror the load source unit However, this 

number 442, or the same I/O address of the primary approach means that the sole source of mirrored state 

load source unit 443, the NVRAM is not considered information when re-initializing is the single load source 

usable. In addition, if the NVRAM contains an invalid 40 unit, losing much of the redundancy offered by mir- 

value for the state of the load source units 444, it is not rored storage. Therefore, it is recommended that the 

usable. user mirror the load source unit 

The Get Stored State routine called at 432 and 433 is The complete Device Configuration Table is stored 

shown in FIG. 4E. If the specified disk unit has reported on all disk units. The copies of the Table on units other 

in, it branches at 451 to return the state that was read 45 than the load source units are not used in the re-initiali- 

from the disk unit when it reported in 452. If the unit has zation procedure described above, and exist only for 

not reported, it checks whether NVRAM is useable archival purposes. In an alternate embodiment, it would 

453. If NVRAM is usable, it returns the state stored in be possible to use these copies of the DCT in place of 
NVRAM 455; otherwise it returns "state unknown" the load source state stored on non-volatile RAM. In a 

454. 50 further alternative embodiment, it would be possible to 
The Validate DCT routine called at 423 is shown in individually determine the state of each mirrored pair of 

FIG. 4F. The routine sets a flag indicating whether the disk units using the same procedure that is used to deter- 
DCT is usable or not. If the load source is not a mir- mine the state of the load source units, rather than rely 
rored pair, the routine immediately branches 461 to set on the DCT in the load source units for the state infor- 
DCT usable. Otherwise, it calls the Get Load Source 55 mation of all other units. Since the units are brought on 
State routine at 462 to obtain the state of the primary line and report in in a random fashion, an additional 
load source device. This is the device from which the alternative embodiment would be to apply the state 
DCT table was originally loaded into memory. If the determination procedure to whichever mirrored pah- 
Get Load Source routine returns a state indicating that reports first, and to then use the state information from 
the primary load source device has current data, the 60 the Device Configuration Table on one of the current 
process branches 463 to set DCT usable; otherwise it units of the first reporting mirrored pair to determine 
sets DCT not usable. It then returns to the calling rou- the states of the remaining mirrored pairs, 
tine. The preferred embodiment stores the alternate state 
The Device Configuration Task shown in FIG. 4A record in non-volatile RAM. This record may altema- 
continues to run indefinitely. Concurrently, the Main 65 tively be stored in any location other than the two load 
IPL Task shown in FIG. 4G is started. After the first source units, provided that the data persists when the 
part of the IPL task 471 completes, the task checks system is powered down. It could, for example, be 
whether all required storage units have reported 472, stored on a third disk storage unit of the same type as 
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the load source units, on tape, or even on some other 
computer system remotely or directly connected to 
computer system 100. 

In an additional alternate embodiment, the serial 
number of the non-volatile RAM unit can be stored in 5 
the non- volatile RAM itself and on the disk units. These 
serial numbers can be compared in the Validate 
NVRAM routine for an additional level of redundancy 
in validating non-volatile RAM. 

Although a specific embodiment of the invention has 10 
been disclosed along with certain alternatives, it will be 
recognized by those skilled in the art that additional 
variations in form and detail may be made within the 
scope of the following claims. In particular, while the 
disclosed preferred embodiment employs magnetic disk 15 
storage units, the invention is applicable to a variety of 
storage device technologies, such at optical disks and 
magnetic tape. 

What is claimed is: 

1. A method for determining true current state of a 
mirrored pair of data storage units in a data processing 
system, said method being performed automatically by 
said data processing system, said method comprising the 
steps of: 25 

attempting to retrieve a first state record from a first 
storage unit of said mirrored pair of storage units to 
obtain a state associated with said first storage unit, 
wherein the state associated with said first storage 
unit includes state information for the mirrored 3Q 
pair; 

attempting to retrieve a second state record from a 
second storage unit of said mirrored pair of storage 
units to obtain a state associated with said second 
storage unit, wherein the state associated with said ^ 
second storage unit includes state information for 
the mirrored pair, and wherein at least one of the 
state associated with said first storage unit and the 
state associated with the second storage unit is 
different from the true current state of said mix- ^ 
rored pair; 

retrieving an alternate state record from a location in 
said data processing system other than said first 
storage unit and other than said second storage 
unit; 45 

deriving a state associated with one of said first or 
second storage units from state information con- 
tained in said alternate state record if it is not possi- 
ble to retrieve the state record from the respective 
storage unit; and 50 

determining the true current state of said mirrored 
pair from tile state associated with said first storage 
unit and the state associated with said second stor- 
age unit. 

2. The method for determining true current state of a 55 
mirrored pair of data storage units of claim 1, wherein 
said step of determining the true current state of said 
mirrored pair comprises retrieving the true current state 
of said mirrored pair from an entry in a state derivation 
table, said state derivation table having a plurality of 60 
entries, each said entry corresponding to a respective 
pair of states, wherein said entry containing said true 
current state corresponds to the states associated with 
said first and second storage units. 

3. The method for determining true current state of a 65 
mirrored pair of data storage units of claim 2, wherein 
said alternate state record is retrieved from non-volatile 
random access memory. 



4. The method for determining true current state of a 
mirrored pair of data storage units of claim 1, wherein 
said alternate state record is retrieved from non- volatile 
random access memory. 

5. A method for determining true current states of the 
mirrored pairs of data storage units in a data processing 
system having at least two mirrored pairs of storage 
units, said method being performed automatically by 
said data processing system, said method comprising the 
steps of: 

attempting to retrieve a first state record from a first 
storage unit of a first mirrored pair of storage units 
to obtain a state associated with said first storage 
unit, wherein the state associated with said first 
storage unit includes state information for the mir- 
rored pair; 

attempting to retrieve a second state record from a 
second storage unit of said first mirrored pair of 
storage units to obtain a state associated with said 
second storage unit, wherein the state associated 
with said second storage unit includes state infor- 
mation for the mirrored pair, and wherein all least 
one of the state associated with said first storage 
unit and the state associated with said second stor- 
age unit is different from the true current state of 
said mirrored pair; 

retrieving an alternate state record from a location in 
said data processing system other than said first 
storage unit and other than said second storage 
unit; 

deriving a state associated with one of said first or 
second storage units from state information con- 
tained in said alternate state record if it is not possi- 
ble to retrieve the state record from the respective 
storage unit; 

determining the true current state of said first mir- 
rored pair from the state associated with said first 
storage unit and the state associated with said sec- 
ond storage unit of said first mirrored pair, said true 
current state indicating which units of said first 
mirrored pair contain current data; and 

retrieving the state of each mirrored pair of storage 
units other than said first mirrored pair from any 
storage unit of said first mirrored pair which is 
determined, by said step of determining the true 
current state of said first mirrored pair, to have 
current data. 

6. The method for determining true current states of 
mirrored pairs of data storage units of claim 5, wherein 
said step of detennining the true current state of said 
first mirrored pair comprises retrieving the true current 
state of said first mirrored pair from an entry in a state 
derivation table, said state derivation table having a 
plurality of entries, each said entry corresponding to a 
respective pair of states, wherein said entry containing 
said true current state corresponds to the states associ- 
ated with said first and second storage units of said first 
mirrored pair. 

7. The method for determining true current states of 
the mirrored pairs of data storage units of claim 6, 
wherein said alternate state record is retrieved from 
non-volatile random access memory. 

8. The method for detennining true current states of 
the mirrored pairs of data storage units of claim 5, 
wherein said alternate state record is retrieved from 
non-volatile random access memory. 
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9. A data storage apparatus for a data processing 
system having a mirrored pair of data storage units, 
comprising: 

a first storage unit of said mirrored pair; 

a second storage unit of said mirrored pair; 5 

means for storing and retrieving a first state record, 
said first state record having state information for 
said mirrored pair, said first state record being 
stored on said first storage unit; 

means for storing and retrieving a second state re- W 
cord, said second state record having state informa- 
tion for said mirrored pair, said second state record 
being stored on said second storage unit wherein at 
least one of the state information in the first state 
record and the state information in the second state 15 
record is different from true current state of said 
mirrored pair; and 

state determination means, accessing said means for 
storing and retrieving a first state record and ac- 
cessing said means for storing and retrieving a 
second state record, for determining automatically 
and without input from an operator the true cur- 
rent state of said mirrored pair of data storage units 
when the data processing system is initialized, ^ 
wherein said state determination means determines 
the true current state of said mirrored pair when 
the state record retrieved from one of the first or 
second storage units indicates that both storage 
units of said mirrored pair contain current data, and 3Q 
it is not possible to retrieve the state record from 
the other one of the first or second storage unit, 

10. The data storage apparatus of claim 9, wherein 
said state determination means comprises means for 
substituting the state information contained in an alter- 35 
nate state record for the state information contained on 
one of the storage units in the event the state informa- 
tion contained on the storage unit can not be retrieved. 

11. The storage apparatus of claim 10, wherein the 
state determination means further comprises a state 40 
derivation table having a unique state entry correspond- 
ing to each possible pair of a state associated with the 
first storage unit and a state associated with the second 
storage unit, whereby the true current state of said 
mirrored pair is determined by retrieving from said state 45 
derivation table the entry corresponding to the state 
associated with said first storage unit and the state asso- 
ciated with said second storage unit. 

12. The data storage apparatus of claim 11, further 
comprising: SO 

a third storage unit for storing data; 

a fourth storage unit for storing a copy of the data 
stored on said third storage unit; 

wherein said state determination means automatically 
determines the states of said third and fourth stor- 55 
age units using the information contained in the 



state records stored on said first and second storage 
units. 

13. A data storage apparatus for a data processing 
system, comprising: 
a first storage unit for storing data; 
a second storage unit for storing a copy of the data 

stored on the first storage unit; 
means for storing and retrieving a first state record, 
said first state record having state information for 
said data storage apparatus, said first state record 
being stored on said first storage unit; 
means for storing and retrieving a second state re- 
cord, said second state record having state informa- 
tion for said data storage apparatus, said second 
state record being stored on said second storage 
unit wherein at least one of the state information in 
the first state record and the state information in 
the second state record is different from true cur- 
rent state of said mirrored pair; and 
alternate state record substitution means, for substi- 
tuting information contained in an alternate state 
record for the information contained in the state 
record of one of the storage units, when the state 
retrieval means is unable to retrieve the state infor- 
mation from the respective storage unit; and 
state determination means, coupled to the means for 
storing and retrieving a first state record and the 
means for storing and retrieving a second state 
record, and accessing the alternate state record 
substitution means, for determining automatically 
and without input from an operator, when the data 
processing system is initialized, the true current 
state of said data storage apparatus from the state 
associated with said first storage unit and the state 
associated with said second storage unit. 
14 The data storage apparatus of claim 13, wherein 
the state determination means further comprises a state 
derivation table having a unique state entry correspond- 
ing to each possible pair of a state associated with the 
first storage unit and a state associated with the second 
storage unit. 

15. The data storage apparatus of claim 14, wherein 
said alternate state record substitution means retrieves 
said alternate state record from non-volatile random 
access memory. 

16, The data storage apparatus of claim 15, further 
comprising: 

a third storage unit for storing data; 

a fourth storage unit for storing a copy of the data 
stored on said third storage unit; 

wherein the state determination means automatically 
determines the states of said third and fourth stor- 
age units using the information contained in the 
state records stored on said first and second storage 
units. 

***** * 
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